DOCUMENT RESUME 



ED 0«3 681 


TM 000 162 


AUTHOR 


Pice, Ernest Thomas 


TITLE 


An Exploratory Study to Investigate the Aspects of 
the Construction and Use of Instruments Enconpassinq 




the Evaluative Strategies of Context, input. 
Process, and Product of In-Service Programs. 


INSTITUTION 


Iowa Univ. , Iowa City. Iowa Center for Research in 
School Administration. 


REPORT NO 


RD-NO-51 


PUB DATE 


Apr 6° 


NOTE 


tip. 


EDRS PRICE 


EDRS Pr<ce HP- $0. 25 HC-J0.30 


descriptors 


Administrative Personnel, Analysis of Variance, 
♦Decision Making, Educational Change, Elementary 
School Teachers, Evaluation, *Tnsorvice Programs, 
♦Instructional Improvement, ♦Models, ♦Program 
Evaluation, Secondary School Teachers 


ABSTRACT 


This paper describes the construction and use of a 



set t f evaluative instruments to aid in the planning of in-service 
proqrams. The four instruments constructed used Stuf iiebeam's four 
evaluative strategies: context, input, process, and product. They 
were designed to provide rating which could be compared in regard to 
personnel grouD and local school district variables. The instruments 
were administered to U10 teachers and 1**3 administrators in Iowa 
school districts. Differences in administrators' and teachers' 
perceptions in evaluating in-service programs was a major factor in 
th° study. Superintendents rated all four concepts higher than 
principals and both grouos rated all four higher than teachers. Older 
teachers gave higher ratings than younger teachers. Other differences 
are noted and interpretations suogested. (IP) 
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Introduction 

From his study of the literature on education, Mr. Rice observed that one of the 
greatest problems facing education today is that of keeping teachers abreast of changing 
conditions. In order for tlw schools to assist their teachers to meet the pressures of 
change and to improve Instruction in-service education is essential. Related to the 
improvement of instruction is the necessity for good evaluation procedures. 

Problem 



The problem of this study was to construct a set of evaluative instillments that 
would be of better use to the decision-maker in planning in-service programs than the 
instruments now In common use . 

The problem focused on the following considerations: 

1 . To construct a set of evaluative instruments, encompassing evaluative strategies of a 
decision-making model, for the purpose of measuring certain categorized elements of 
in-service problems . The model which was used was that of Daniel Stufflebeam 

2 . To obtain ratings from a sample and view them In relation to: 

a. The variables of context, input, process and product 

b. The variables associated with clashes of personnel: 

Administrative personnel vs. teacher personnel; 
elementary personnel vs. secondary personnel; 
older vs. younger teachers; teac^rs with four 
years of college training or less . . teachers with 
five or more years of college training. 

c. The variables associated with local school district: size of enrollment, 

assessed value per pupil average number of years that teachers have 
remained in the district . 

The general purpose of the investigations was to study the characteristics of the > 
instruments which were constructed by the author and to make suggestions for their 
future use and modification. 

A more speclUc purpose of the study was to make suggestions that could be helpful 
to those Interested in improving an in-service program. A question which was particularly 
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important for this study was, "llov/ do the perceptions of administrators compare tc the 
perceptions of teachers in the evaluation of various aspects of the in-service program?" 

In reviewing the literature on the subject of evaluation, the author concluded that 
recent efforts of evaluation appear to be moving away from external evaluation of teacher 
traits and school activities and toward giving more emphasis to selfrevaluation of the parts 
of the school program and the situation in which these parts exist. V/hat is being evaluated 
and for what purpose the evaluation is being made must be carefully spelled out before an 
evaluation is undertaken. 

In general the review of the literature centered around the most critical needs for 
professional growth, criteria for evaluating in-service programs, tr/t methods used by 
researchers to evaluate in-service programs, and the needs of decisions -makers in an 
atmosphere of change . 

Methods 

Four instruments were constructed. Each one related to one of the four evaluative 
strategies proposed by Stufflcbcam, that is the context, input, process, and product of 
evaluation. The major objective of context evaluation was to define the environment’s 
unmet needs, and the problems underlying those needs. Input evaluation was to provide 
information for deciding whether outside assistance was to be sought for meeting goal3 and 
objectives, and what strategies were to be employed to reach the proposed goals . Process 
evaluation sought to predict or detect, during the implementation stages, defects in the 
procedural design. Product evaluation wa 3 used to determine the effectiveness of the pro- 
ject after it had completed its cycle, or, in other words, to measure and interpret the 
outcomes . 

The four instruments were used in a sutvey of the sixty school districts in Area IX 
and Area X in East Central Iowa. The sample consisted of 173 administrative personnel 
end 410 teachers. A three -factor Lindq .1st Type III analysis of variance design was used 
among otter types of statistical tests. 



Findings 



On the basis of the instrument used, the writer found that there existed some 
difference between the administratis personnel and the teachers concerning the oncepts 
involved. There was, however, no significant difference between the results of the four 
types of districts studied. Superintendents rated all four concepts significantly higher than 
principals, and both administrative groups rated all four concepts (context, process, Input 
and product) iilgher than teachers. Elementary and secondary teachers rated alt four 
concepts the same. Older teachers gave higher ratings on the four concepts than younger 
teachers . 
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Conclusions 

i 

Of the four concepts there was less discrepancy indicated in the ratings between adminis- 
trators and teachers on context and more discrepancy indicated on input, process and 
product . Although it was difficult to infer which of the four strategics administrators or 
teachers are best able to evaluate , there appeared to be considerable vaiue in comparing 
discrepancies between administrators and teachers on the four strategies as a diagnostic 
measure in order to plan cooperative activities between teachers and administrators. Some 
probable benefits resulting from this approach would be the development of a common 
philosophy toward ir -service education and better participation in shared decision-making 
in order to agree upon objectives and better means of ensuring effective in-service 
activities . ? 

h was found in this study that there was a significant difference among the four 
concepts as rated by teachers. There was also a significant difference between the con- 
cepts as rated by administrators on the four instruments. Context was rated significantly 
luglicr than input, piocess, and product by both teachers and administrators. Product was 
rated significantly higher than inpjt and process by teachers only. This tends to infer 
that administrators feel that tlie district provides a climate, environment and cooperation 
that is capable of providing a better in-service program than is indicated hy tire aspects 
of organization, implementation, end achievement of objectives, if the instruments arc 
measuring what they were intended to ueasv.:re. It could also be interpreted tha'. .'eachcrs 
feel the same way, except tliat they rate product higher in relation to the other three 
concepts than administrators do. Tills would appear to indicate that teachers have more 
confidence in the outcomes of their in-service programs in relation to the environment, 
organization and implementation than administrators do. 

The fact that no significant difference was found between the four different districts 
used in this study may be due to tlie confounding of size of enrollment with wdaith. Altliough 
it could be suggested that tcacliers would be more content with their in-service programs 
in districts that retained their teacliers longer, this was not born out by this study. Tlie 
fact t'uat older teachers rated their programs significantly higher than younger teachers 
may be due to cognitive dissonance . Altliough one previous study found out that, teachers 
with a 3achclor's degree rated tlieir programs higher than teachers with five or more 
years of college, no significant difference was indicated in this study. Elementary and 
secondary teachers appeared to rate tlieir in-service programs the same 

Tlie author concludes that there appears to be value In using such instruments again 
after revisions are made and carried out in order to analyze changes in ratings given to 
items indicating weaknesses and to indicate whether administrators and teachers are mote 
in agreement on their ratings. Less discrepancy would show Improvement in the 
accuracy of perceptions between levels of personnel . 

Norms were developed for teacher and administrators on all four strategies. These 
could be used to determine whet’.ier tliere was a "normal" discrepancy between teachers 
and administrators in a district and to infer how the district rated in relation to other 
districts in the sample on each of tlie four strategies. 
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